AI-Enhanced Auto-Correction of Programming Exercises: How Effective is GPT-3.5?

Timely formative feedback is considered as one of the most important drivers for effective learning. Delivering timely and individualized feedback is particularly challenging in large classes in higher education. Recently Large Language Models such as GPT-3 became available to the public that showed promising results on various tasks such as code generation and code explanation. This paper investigates the potential of AI in providing personalized code correction and generating feedback. Based on existing student submissions of two different real-world assignments, the correctness of the AI-aided e-assessment as well as the characteristics such as fault localization, correctness of hints, and code style suggestions of the generated feedback are investigated. The results show that 73% of the submissions were correctly identified as either correct or incorrect. In 59% of these cases, GPT-3.5 also successfully generated effective and high-quality feedback. Additionally, GPT-3.5 exhibited weaknesses in its evaluation, including localization of errors that were not the actual errors, or even hallucinated errors. Implications and potential new usage scenarios are discussed

Final 3-Year Outcomes of a Randomized Trial Comparing a Self-Expanding to a Balloon-Expandable Transcatheter Aortic Valve.

BACKGROUND In the SCOPE I trial (Safety and Efficacy of the Symetis ACURATE Neo/TF Compared to the Edwards SAPIEN 3 Bioprosthesis), transcatheter aortic valve implantation with the self-expanding ACURATE neo (NEO) did not meet noninferiority compared with the balloon-expandable SAPIEN 3 (S3) device regarding a composite end point at 30 days due to higher rates of prosthetic valve regurgitation and acute kidney injury. Data on long-term durability of NEO are scarce. Here, we report whether early differences between NEO and S3 translate into differences in clinical outcomes or bioprosthetic valve failure 3 years after transcatheter aortic valve implantation. METHODS Patients with severe aortic stenosis were randomized to transfemoral transcatheter aortic valve implantation with NEO or S3 at 20 European centers. Clinical outcomes at 3 years are compared using Cox proportional or Fine-Gray subdistribution hazard models by intention-to-treat. Bioprosthetic valve failure is reported for the valve-implant cohort. RESULTS Among 739 patients, 84 of 372 patients (24.3%) had died in the NEO and 85 of 367 (25%) in the S3 group at 3 years. Comparing NEO with S3, the 3-year rates of all-cause death (hazard ratio, 0.98 [95% CI, 0.73-1.33]), stroke (subhazard ratio, 1.04 [95% CI, 0.56-1.92]), and hospitalization for congestive heart failure (subhazard ratio, 0.74 [95% CI, 0.51-1.07]) were similar between the groups. Aortic valve reinterventions were required in 4 NEO and 3 S3 patients (subhazard ratio, 1.32 [95% CI, 0.30-5.85]). New York Heart Association functional class ≤II was observed in 84% (NEO) and 85% (S3), respectively. Mean gradients remained lower after NEO at 3 years (8 versus 12 mm Hg; P<0.001). CONCLUSIONS Early differences between NEO and S3 did not translate into significant differences in clinical outcomes or bioprosthetic valve failure throughout 3 years. REGISTRATION URL: https://clinicaltrials.gov, Unique identifier: NCT03011346